In Use, In Situ: Extending Field Research Methods

Many authors have argued the need for a broader understanding of context and the situatedness of activity when approaching the evaluation of systems. However, prevailing practice often still tends towards attempting to understand the use of designed artefacts by focusing on a core set of tasks that are thought to define the system. A consequence of such focus is that other tasks are considered peripheral and outside the scope of design and evaluation activities. To illustrate the point, consider the experience, familiar to many of us, of being involved in an evaluation activity where participants provide unstructured qualitative feedback. Irrespective of whether the activity is carried out in a laboratory, in a high fidelity simulation or in a naturalistic setting, participants will frequently volunteer unsolicited feedback about tasks and goals that were not originally within the ambit of the design activity. This unprompted feedback, we suggest, is a cue for the evaluators to pay attention to the relationship between the tool and the practice in which it will be used. In other words a cue to consider the situations in which artefact will be used, the tasks and activities that may be affected by the new system, and so on. These are empirical questions that cannot be answered a priori by the development team, whether the evaluation is taking place in “artificial” or “natural” setting.